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Using PIRLS Data to Investigate the Relationship of 
Teachers’ Instruction with Students’ Out-of-School 
Reading Behaviors 

Summary 

Research clearly demonstrates the importance of students’ reading 
behaviors for predicting students’ short- and long-term outcomes. 
While teachers’ instruction might affect these reading behaviors (and 
therefore indirectly affect achievement), we know little about the 
association of in-school teacher practices with students’ out-of-school 
behaviors. In this brief, we draw on data from lEA’s 201 1 Progress 
in International Reading Literacy Study (PIRLS) to examine the 
relationships of several instructional practices with multiple measures 
of students’ out-of-school reading. Finding wide variation in instruction 
across countries and several significant associations, we conclude 
with policy implications for policymakers, teachers, parents, and 
researchers. 

Introduction 

Reading achievement at an early age is a strong predictor of future 
grades, educational attainment (Cunningham & Stanovich, 1997; 
Duncan et al . , 2007; McGee, Prior, Williams, Smart, & Sanson, 

2002), and even socioeconomic status in adulthood (Ritchie 
& Bates, 2013). One major contributor to children’s reading 
achievement is the amount that children read (both frequency and 
variety of topics; see, for example, Cox & Guthrie, 2001 ). In addition 
to directly affecting reading achievement, amount of reading (in 
this case measured by familiarity with popular authors) benefits 
children by enriching their knowledge of the world (e.g., Stanovich 
& Cunningham, 1993). 

Students’ in-school reading activities are clearly directly affected 
by teachers’ instruction, which is itself directly influenced by 
instructional policies such as school, district, state, regional, or 
national curriculum policies. In contrast, literature showing that 
education policies directly affect students’ out-of-school reading 
behaviors is rare. However, educational policies could have indirect 
effects on students’ out-of-school reading behaviors if educational 
policies influence teachers’ instruction or parental behaviors, which 
in turn influence students’ reading behaviors both in and out of 
school. Figure 1 provides a simplified conceptual model of these 
influences. 
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Figure 1: Simplified conceptual model for the influence of education policies on student reading 
knowledge and skills 



In this study, we examine one path by which 
educational policies might influence students’ 
reading achievement — through the influence of 
teachers’ instruction (curriculum and pedagogy) 
on students’ out-of-school reading behaviors (i.e., 
we test the relationship signified by the arrow 
connecting those two boxes in Figure 1). Our 
study uses data from fourth graders participating 
in the 2011 Progress in International Reading 
Literacy Study (PIRLS 2011). 

Three research questions guide our work: 

1 . To what extent do students vary in their 
out-of-school reading, both within and 
between countries? 

2. To what extent do countries vary in their 
instructional practices associated with 
reading? 

3. What are the associations of in-school 
instructional practices with out-of-school 
reading behaviors? 

Our findings suggest hypotheses to be tested 
using more rigorous designs regarding the 
influence of instruction on student reading 
behaviors, and thus their reading achievement. To 
the extent that instructional practices are affected 
by educational policies, the results also point the 
way toward policies that may positively influence 
students’ reading behaviors. 

Data and Methods 

PIRLS 201 1 surveyed over 50 education systems 
(Foy, 2013). Our sample excludes benchmarking 
regions 1 and countries that did not survey fourth 
graders 2 . Our final sample has 44 education 
systems and 244,41 1 students. 

The dependent variables (four in total) that we use 
to measure out-of-school reading come from the 
PIRLS student surveys. Variable 1 is the amount 
that students read outside of school every day, 
measured in hours. The remaining three variables 
concern the purpose of reading outside of school, 


measured in per-day units (e.g., 1 = every day, 

5 = every two days). Variable 2 is the frequency 
of reading for fun, Variable 3 is the frequency of 
reading self-chosen books, and Variable 4 is the 
frequency of reading to learn. 

For in-class practices, we construct five variables 
based on the PIRLS teacher surveys. To assemble 
these scales, we used principal components 
analyses with varimax rotation in order to identify 
common factors. Each scale has a 0 to 1 range 
where 1 = every day. The five variables are 1 ) 
using fiction or nonfiction text (type of text), (2) 
reading aloud to the class or asking students to 
read out loud (reading aloud), (3) asking students 
to read self-chosen books or to read silently in 
class (independent reading), (4) conducting 
other reading activities (reading activity), and 
(5) developing reading strategies (reading 
strategies). 

Some of these scales have relatively low 
reliabilities due to the small number of items 
for the scales. If the reliability of these scales 
is low due to measurement error, that will tend 
to depress the correlations of these measures 
with our outcomes. Thus, we view the results we 
identify in this paper as conservative estimates 
of what would be found if more comprehensive 
scales could be constructed. Regardless, the 
results are stronger than if we used single-item 
instructional indicators, which would of course 
have even less reliability than our scales. 

For Research Questions 1 and 2, we use 
descriptive statistics to compare country-level 
averages. We also partition the variance in these 
measures within and between countries using a 
simple two-level model (students or classrooms 
nested in countries). For Research Question 3, we 
conduct multilevel regression analyses to examine 
the relationships between teachers’ instruction 
and our four out-of-school reading outcome 
variables. 
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Regression analysis allows us to control for 
student, family, class, and school characteristics 
that can also affect students’ reading habits. We 
use three-level models, with students nested 
within classes/schools nested within countries. 

Our models use random effects for classes 
because our focal independent variables (i.e. , 
instructional practices), are class-level variables. 
We also use country fixed effects (i.e., dummy 
variables, one for each country) to control for all 
country-level predictors of students’ out-of-school 
reading. Thus, our regression results are based on 
average within-country relationships of instruction 
with out-of-school reading. 

Findings 

To what extent do students vary in their out-of- 
school reading? 

Table 1 (see Table 1 ; all tables are in appendix) 
shows the country-level averages of the four out- 
of-school reading variables. On average, fourth 
graders across the 44 countries report spending 
0.84 hours (about 50 minutes) reading out of 
school every day. Country-level averages vary 
somewhat. Students in Iran report spending the 
most time — 1 .28 hours per day — reading outside 
of school, almost half an hour longer than the 
international average. In contrast, students from 
northern European countries such as Denmark, 
Norway, and Sweden, spend closer to 0.6 hours 
reading every day. Students from the U.S. spend 
0.78 hours per day, slightly below the international 
average. 

Fourth graders from most education systems 
report reading every other day for fun or to learn, 
and they read a book of their own choice on two 
out of every three days on average. We found 
some interesting exceptions to these averages, 
however. For instance, Azerbaijani students report 
reading outside of school for fun on just over 
one-quarter of days (0.28 times/day) and reading 
to learn on three-quarters of days (0.75 times/ 
day). At the other extreme, students in northern 
European countries such as the Netherlands and 
Finland report reading to learn on just a third of 
days (0.32 and 0.33 times/day). Students from 
some eastern and southeastern Asian countries 
such as Hong Kong SAR, Chinese Taipei, and 
Singapore are also among those who read the 
least frequently for fun (0.44, 0.41 , and 0.39 times/ 
day, respectively). Students from the U.S. are 
slightly more likely than the international averages 
to read a book of their choice and slightly less 
likely to read to learn. 

Table 2 partitions the variance in reading habits 


into within- and between-country sources based 
on a simple two-level model (students nested 
in countries). It shows that the majority of the 
variation in these outside-of-school reading 
measures occurs within countries (92% to 98%, 
depending on the measure). This means that 
students from different countries are not very 
different from one another in their reading habits 
outside of school. It also suggests that country- 
level variables are not likely main drivers of 
differences in out-of-school reading behaviors. 

To what extent do countries vary in their 
reading instructional practices? 

We now turn to a descriptive examination of 
reading instruction across countries. Table 3 
shows the country-level averages of instructional 
variables. Variation in instructional practices 
across countries is much larger than the variation 
in students’ out-of-school reading behaviors. For 
instance, Grade 4 teachers in Iran and Belgium 
report spending less than one day in six using 
fiction or nonfiction materials in reading instruction 
(both 0.15 times/day), while teachers in Sweden 
and the U.S. report using fiction or nonfiction texts 
roughly 150 percent more often (0.36 and 0.37 
times/day). 

Reading aloud is the most common instructional 
activity among those measured in our study. 

On average across the 44 countries, Grade 4 
teachers report reading out loud in class four out 
of every five days (0.75 times/day). In half of the 
education systems, teachers read aloud almost 
every day (more than 0.75 times/day). Teachers in 
only a handful of countries (i.e., Austria, Chinese 
Taipei, and Denmark) report reading aloud less 
than every other day. 

In terms of independent reading in class, 
teachers from Australia, Canada, New Zealand, 
and the U.S. report allowing students to read 
independently most often in class — roughly 9 
out of every 10 days. In contrast, students in Iran 
and the Czech Republic have considerably less 
opportunity to read independently in class — less 
than two out of every five days. 

Fourth-grade teachers from Denmark, Finland, 
Sweden, and Belgium are among those least 
likely to use other reading activities. They report 
conducting other reading activities only one out of 
every five days. At the other extreme, Azerbaijani 
students experience other reading activities nearly 
every day. 
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Finally, countries also vary in terms of in-class 
development of reading strategies. On average, 
teachers report helping their Grade 4 students 
develop reading strategies about every other 
day (0.56 times/day). However, on the high 
end, teachers from Georgia and Israel say they 
have students engage in activities designed to 
develop reading strategies on roughly 80 percent 
of teaching days; teachers from Sweden and 
Belgium, on the other hand, say they use such 
activities on between one-quarter and one-third 
ofdays (0.29 and 0.27 times/day). 

The results from our two-level model also confirm 
substantial variation in the teachers’ reported 
instructional variables between countries — far 
more than for the students’ reported out-of- 
school reading behaviors. Grade 4 teachers 
from different education systems differ markedly 
in terms of helping students develop reading 
strategies, with 31 .53 percent of the variation in 
this variable coming from country-level differences 
(see Table 4). For the other instructional 
variables, the proportion of the variance found at 
the country level ranges from 16.55 percent to 
26.72 percent. 

What are the associations of in-schooi 
instructional practices with out-of-school 
reading behaviours? 

Table 5 shows the coefficients of teachers’ 
instructional practice variables from multi-level 
regression analyses. The full results appear 
in the online supplement. 3 - 4 In this section, we 
briefly discuss the statistically significant results 
(p<0.05). 

With respect to time spent on reading, only one 
of the five variables is a statistically significant 
predictor. Fourth graders in classes where 
teachers report using fiction or nonfiction texts 
are the students who are likely to spend more 
time reading outside of school. These coefficients 
are quite small, however. The difference between 
the highest and the lowest countries on using 
fiction or nonfiction texts in class is 22 percentage 
points. Our model predicts that this difference 
would be associated with about a five-minute-per- 
week difference in the amount of time students in 
those countries spend reading outside of school. 
Two statistically significant instructional predictors 
are evident with regard to the outcome variable 
‘frequency of fourth graders reading for fun’. 
Specifically, reading fiction and nonfiction texts 
in class is positively associated with students 
reading for fun outside of school, but developing 
reading strategies in class is negatively 


associated with reading for fun outside of 
school. For reading fiction and nonfiction texts, 
the difference between the highest and lowest 
countries of 22 percentage points predict a 0.4 
percent difference in the frequency of reading 
for fun. The coefficient for reading strategies 
is roughly the same size but in the opposite 
direction. 

When the outcome variable is ‘frequency of 
reading a book of one’s choice’, the only one 
statistically significant predictor is, not surprisingly, 
the frequency of students’ independent reading 
in class. Indeed, this is the strongest predictor 
(in terms of statistical significance) of any in our 
models. The difference between the highest 
and lowest countries on independent reading 
in school is about 53 percentage points, so our 
model predicts that this difference is associated 
with about a 1.1 percentage-point difference in 
the frequency of reading chosen books outside of 
school. 

For the final outcome, ‘reading to learn outside of 
school’, the only statistically significant predictor is 
the frequency of reading activities in class. Again, 
the coefficient is of a similar magnitude. Given 
that the difference between the highest and lowest 
countries on this measure was quite large — 67 
percentage points — the coefficient suggests 
this difference predicts a 1.1 percentage-point 
difference in the frequency of reading to learn 
between these two countries. 

In summary, our regression results suggest 
several intuitive relationships of in-class 
instructional behaviors with out-of-school reading 
behaviors among fourth graders. For instance, 
students reported more frequently reading chosen 
books outside of class when their teachers more 
frequently incorporate independent reading in 
class. This finding is consistent with previous 
research in the U.S. (Gambrell, 1996; Guthrie, 
Wigfield, & Perencevich, 2004; Wigfield & Guthrie, 
2000). Also, students are more likely to report 
reading to learn when they are exposed to more 
reading activities in class. On the other hand, 
teaching reading strategies is associated with less 
frequent reading for fun among fourth graders. 
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Discussion 

Our study finds overall that the PIRLS 201 1 fourth 
graders around the world do not dramatically 
differ in terms of their out-of-school reading habits. 
In contrast, the Grade 4 teachers across the 44 
countries in our sample differ considerably with 
respect to their reported instructional practices. 
This finding suggests policies have a stronger 
influence on teachers’ instruction than on 
students’ behaviors. Still, most of the variation in 
both instruction and student reading behaviors 
lies within countries, suggesting a possible source 
of inequality that policymakers interested in equity 
should carefully consider. 

We also find several associations between 
teachers’ instructional behaviors and students’ 
out-of-school reading behaviors. These 
associations should be investigated further using 
more rigorous designs, but they suggest that 
educational policies may be able to influence 
students’ out-of-school reading behaviors if they 
can affect teachers’ instruction. If we believe that 
out-of-school reading is important, this work might 
lead to hypotheses as to how to improve students’ 
reading in that context. 


In addition to the above findings, our work also 
demonstrates a novel use of the PIRLS data. By 
combining the rich student- and teacher-level 
data offered through the PIRLS database, we 
were able to answer an interesting question that 
is not answerable through use of many other 
data sources, and we were able to do so in an 
international context that may promote hypotheses 
for future research. While these datasets have 
various limitations (e.g., the absence of student- 
level longitudinal data and the relative coarseness 
of the instructional measures), they often include 
a wealth of important variables that researchers 
could use for descriptive analyses and for 
developing and then testing hypotheses via more 
rigorous designs. 



Endnotes 

1 Benchmarking entities included Alberta, Ontario, Quebec, Southern Africa, Malta (Maltese), Abu Dhabi(UAE), Dubai(UAE), and Spain(Andalucia). 

2 Botswana, Honduras, Morocco, and Kuwait surveyed sixth graders. 

3 Despite the large number of independent variables, there is little evidence that multicollinearity is an issue, especially for our focal independent variables. 
For instance, the variance inflation factor (VIF) for our five instructional variables is no greater than two. A common rule of thumb is that the VIF must be 
above 10 before multicollinearity is a concern. The only independent variables with high VIFs are those indicating school location (urban, suburban, town, 
mid-city, and rural). This is not surprising since sets of dummy variables are often highly correlated among themselves, and the effect of multicollinearity 
on estimates should only be seen on the variables that are multicollinear 

4 Additional regression table can be found in Supplement 1 online at: 
http://www.iea.nl/fileadmin/user_upload/Policy_Briefs/IEA_Policy_Brief_Oct2015_Supplement.pdf 
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Recommendations 


We conclude with brief and non-exhaustive recommendations for researchers , 
policymakers, teachers, interested citizens, parents, and others. 


Policy Makers 

• Consider the ways in which instructional 
policies might influence outcomes other than 
reading achievement, or consider the ways 
that policies might indirectly affect desired 
outcomes. 

• Design data collections to gauge the extent 
of inequality in both in-school and out-of- 
school behaviors that might contribute to 
student achievement. 

Teachers 

• To the extent that these results are supported 
by more rigorous research, consider 
incorporating more independent reading in 
class to help students to establish the habit 
of reading independently. 

• Conduct instructional activities that stimulate 
students’ interest in reading and make in- 
class reading a more enjoyable activity that 
encourages out-of-class reading. 


Researchers 

• Consider building more comprehensive 
measures of theoretically important 
instructional behaviors into future national and 
international studies. 

• Explore more complete conceptual/theoretical 
models of the influence of policy or instruction 
on student behaviors and outcomes, perhaps 
using structural equation models or other 
sophisticated modeling approaches. 

• Design studies to test the associations 
identified here, using more rigorous 
methodologies that can identify causal 
relationships 

Parents and interested citizens 

• Ensure students have access to the materials 
(e.g., books, technology) they need in order to 
have equal opportunity to read at home. 

• Consider the ways that parents and nonschool 
agencies can support the educational efforts 
that schools take to improve student reading 
behaviors. 
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Appendix 


Table 1: Average extent of student reading outside school by country 


Country 

Time spent on reading outside 
school (hours/day) 

Read for fun 
(times/day) 

Read to learn 
(times/day) 

Read book of one’s 
choice (times/day) 

Azerbaijan, Republic of 

0.89 

0.28 

0.75 

0.63 

Australia 

0.79 

0.54 

0.43 

0.70 

Austria 

0.95 

0.60 

0.61 

0.73 

Bulgaria 

0.99 

0.52 

0.65 

0.64 

Canada 

0.78 

0.54 

0.46 

0.73 

Chinese Taipei 

0.82 

0.41 

0.38 

0.51 

Colombia 

0.84 

0.57 

0.77 

0.68 

Croatia 

0.81 

0.42 

0.55 

0.49 

Czech Republic 

0.88 

0.55 

0.54 

0.65 

Denmark 

0.64 

0.55 

0.42 

0.58 

Finland 

0.80 

0.56 

0.32 

0.64 

France 

0.83 

0.56 

0.51 

0.70 

Georgia 

1.08 

0.51 

0.85 

0.74 

Germany 

0.97 

0.59 

0.52 

0.71 

Hong Kong SAR 

0.79 

0.44 

0.40 

0.50 

Hungary 

0.88 

0.51 

0.58 

0.61 

Indonesia 

0.65 

0.65 

0.77 

0.66 

Iran, Islamic Republic of 

1.28 

0.50 

0.67 

0.59 

Ireland 

0.87 

0.56 

0.51 

0.71 

Israel 

0.95 

0.57 

0.58 

0.67 

Italy 

0.70 

0.46 

0.60 

0.60 

Lithuania 

0.88 

0.54 

0.60 

0.71 

Malta 

0.81 

0.56 

0.64 

0.72 

Oman 

0.82 

0.59 

0.63 

0.60 

Netherlands 

0.80 

0.52 

0.33 

0.63 

New Zealand 

0.82 

0.58 

0.49 

0.73 

Norway 

0.61 

0.45 

0.42 

0.64 

Poland 

0.84 

0.50 

0.57 

0.53 

Portugal 

0.60 

0.59 

0.69 

0.74 

Qatar 

0.91 

0.57 

0.63 

0.62 

Romania 

0.94 

0.58 

0.71 

0.68 

Russian Federation 

0.91 

0.58 

0.63 

0.68 

Saudi Arabia 

0.80 

0.56 

0.56 

0.59 

Singapore 

0.93 

0.39 

0.54 

0.66 

Slovak Republic 

0.84 

0.45 

0.55 

0.57 

Slovenia 

0.84 

0.50 

0.59 

0.69 

Spain 

0.79 

0.56 

0.66 

0.70 

Sweden 

0.62 

0.52 

0.43 

0.65 

Trinidad and Tobago 

0.87 

0.50 

0.70 

0.66 

United Arab Emirates 

0.91 

0.56 

0.66 

0.64 

United States 

0.78 

0.49 

0.49 

0.69 

England 

0.81 

0.49 

0.38 

0.67 

Northern Ireland 

0.88 

0.52 

0.43 

0.69 

Belgium (French) 

0.84 

0.58 

0.55 

0.70 

Average 

0.84 

0.52 

0.56 

0.65 
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Table 2: Partitioning the variance in out-of-school reading behaviors 



Between-Country Variation 
(%> 

Within-Country Variation 
(%) 

Reading time 

2.91 

97.41 

Read for fiin 

2.64 

97.78 

Read chosen book 

2.70 

97.46 

Read to learn 

8.84 

92.29 


Table 3: Average extent of use of reading-related instructional practices by country (times/day) 

Use Fiction or Read Read Self-Chosen Reading Develop Reading 

Country Nonfiction Aloud Book Activity Strategy 

Azerbaijan, Republic of 

0.22 

0.89 

0.43 

0.86 

0.75 

Australia 

0.35 

0.72 

0.89 

0.47 

0.49 

Austria 

0.23 

0.45 

0.60 

0.31 

0.37 

Bulgaria 

0.21 

0.96 

0.58 

0.59 

0.76 

Canada 

0.33 

0.71 

0.88 

0.37 

0.51 

Chinese Taipei 

0.19 

0.40 

0.56 

0.33 

0.34 

Colombia 

0.28 

0.72 

0.41 

0.53 

0.54 

Croatia 

0.17 

0.89 

0.42 

0.51 

0.67 

Czech Republic 

0.18 

0.60 

0.38 

0.43 

0.43 

Denmark 

0.24 

0.38 

0.65 

0.23 

0.36 

Finland 

0.24 

0.62 

0.58 

0.19 

0.35 

France 

0.22 

0.79 

0.66 

0.31 

0.38 

Georgia 

0.18 

0.86 

0.42 

0.69 

0.83 

Germany 

0.18 

0.55 

0.53 

0.31 

0.41 

Hong Kong SAR 

0.16 

0.74 

0.52 

0.54 

0.48 

Hungary 

0.25 

0.73 

0.57 

0.46 

0.71 

Indonesia 

0.23 

0.62 

0.41 

0.41 

0.55 

Iran, Islamic Republic of 

0.15 

0.72 

0.37 

0.48 

0.45 

Ireland 

0.28 

0.82 

0.72 

0.41 

0.55 

Israel 

0.31 

0.86 

0.68 

0.68 

0.79 

Italy 

0.24 

0.82 

0.44 

0.59 

0.59 

Lithuania 

0.19 

0.76 

0.50 

0.73 

0.76 

Malta 

0.23 

0.79 

0.57 

0.39 

0.43 

Oman 

0.17 

0.92 

0.44 

0.50 

0.60 

Netherlands 

0.35 

0.75 

0.83 

0.38 

0.34 

New Zealand 

0.33 

0.64 

0.90 

0.46 

0.56 

Norway 

0.31 

0.70 

0.73 

0.29 

0.36 

Poland 

0.35 

0.87 

0.49 

0.63 

0.70 

Portugal 

0.30 

0.89 

0.64 

0.66 

0.75 

Qatar 

0.27 

0.84 

0.50 

0.54 

0.61 

Romania 

0.18 

0.87 

0.61 

0.55 

0.72 

Russian Federation 

0.31 

0.82 

0.64 

0.75 

0.76 

Saudi Arabia 

0.19 

0.90 

0.48 

0.46 

0.58 

Singapore 

0.20 

0.60 

0.69 

0.35 

0.51 

Slovak Republic 

0.20 

0.72 

0.44 

0.51 

0.61 

Slovenia 

0.25 

0.64 

0.52 

0.39 

0.47 

Spain 

0.32 

0.90 

0.60 

0.50 

0.56 

Sweden 

0.36 

0.59 

0.82 

0.20 

0.29 

Trinidad and Tobago 

0.32 

0.89 

0.79 

0.51 

0.58 

United Arab Emirates 

0.25 

0.83 

0.42 

0.49 

0.58 

United States 

0.37 

0.77 

0.89 

0.53 

0.71 

England 

0.26 

0.64 

0.77 

0.37 

0.45 

Northern Ireland 

0.27 

0.74 

0.72 

0.46 

0.43 

Belgium (French) 

0.15 

0.69 

0.58 

0.19 

0.27 

Average 

0.26 

0.75 

0.61 

0.49 

0.56 
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Table 4: Partitioning the variance in reading instructional practices 



Between-Country Variation 
(%) 

Within-Country Variation 
(%) 

Use fiction or nonfiction 

16.55 

83.45 

Read aloud 

22.66 

77.34 

Read self-chosen book 

24.94 

75.06 

Reading activity 

26.72 

73.28 

Develop reading strategy 

31.52 

68.48 


Table 5: Multilevel regressions of 

instructional practice 

es on reading habits outside school 



Time Spent on 

Read for Fun 

Read Book of One’s 

Read to 


Reading 


Choice 

Learn 

Use fiction or nonfiction 

0.053* 

0.019* 

0.012 

0.013 

Read aloud 

-0.009 

-0.003 

-0.004 

-0.002 

Read self-chosen book 

-0.011 

0.003 

0.021*** 

-0.008 

Reading activity 

0.001 

0.010 

-0.002 

0.016** 

Develop reading strategy 

0.009 

-0.016* 

0.001 

-0.004 


Note: p = *< 0.05, **< 0.01, ***< 0.001. Values are coefficients from multilevel regressions. 
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